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This study was undertaken to examine the influ¬ 
ence that a set of Professional Development 
(PD) initiatives had on faculty use ofMoodle, 
a well-known Course Management System. The 
context of the study was a private university in 
Japan that specializes in languages. Specifically, 
it aimed to identify the way in which the PD ini¬ 
tiatives adhered to professional development 
best practice criteria, and what impact the 
PD initiatives had on faculty use of Moodle. 
The study utilized a mixed-methods approach. 
Participants in the study were 42 teachers who 
worked at the university. The results indicated 
that the PD initiatives adhered closely to crite¬ 
ria posited in technology-related professional 
development best practice criteria. Further, 
results from the online survey and follow up 
face-to-face interviews indicated that while the 
PD initiatives that were implemented were pos¬ 
itively perceived by faculty, they did not have 
the anticipated impact on Moodle use. 
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Introduction 


Regular Paper 


The context of this study was a private 
university in Japan that specializes in lan¬ 
guages. The university has used the Moodle 
CMS since 2003 to assist in various language 
related classes such as Academic Writing, 
Intensive Reading, Extensive Reading, 
Freshman English, Communicative 
Grammar, and various content based classes 
for 3rd and 4th year students. An ongoing 
financial commitment has been made to 
include Moodle as an important part of the 
educational landscape within the university, 105 
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and it is hoped that Moodle is used at an individual and departmental level. Each year in 
late March, during training and orientation for new teachers, a one day Moodle orientation 
is provided. During that presentation, teachers are taught how to login to Moodle and to 
create their own classes and add content. However, it was deemed insufficient by both the 
trainers and the new teachers, not least of all because one day cannot show teachers the 
various uses of Moodle, or the many practical ways in which Moodle may be relevant for 
individual faculty members teaching a variety of classes. 

Therefore, the supervisor of the university's technology committee was given the mis¬ 
sion of implementing a thorough professional development (PD) program that could show 
faculty not only how to use Moodle and all its features, but also why they would use it and 
how it could benefit them in the short term and long term paths of their teaching careers. 
The members of the technology committee were all experienced users of Moodle, and were 
required to use Moodle in their classes. There were five members in total, positioned in 
different buildings deliberately to ensure that teachers in different buildings had access to 
Moodle support staff. In addition, one of the requirements of the PD program was to ensure 
that teachers learning about Moodle could access the PD initiatives both on campus and 
off campus whenever they needed to. 

In part, professional development (PD) is designed to improve teaching practice and 
provide fresh impetus on ways to bridge the divide between theory, pedagogy, and practice. 
Although PD initiatives are not directed primarily at students, they are its final beneficiaries 
(Bliss & Bliss, 2003), and an impact on classroom practice is typically the ultimate intended 
outcome of PD (Harland & Kinder, 1997). However, the most effective way to deliver PD in an 
educational setting is still open to debate, particularly in regards to PD and technology in 
education. One way that technology-related PD is commonly delivered is through in-service 
training. Therefore, in that context there needs to be someone within an organization who 
understands effective PD, and the techniques necessary to deliver it. 

That is where a change-agent becomes relevant. According to Pathak (2010), a change- 
agent is defined as a person who acts as a catalyst and assumes the responsibility for 
managing change activities in an organization, and works for the accomplishment of the 
goals or objectives of the organization. Indeed, according to Everett Rogers’ “Diffusion of 
Innovations" model (2003), the “extent of a change-agent's promotion efforts” is one of the 
five most influential variables in terms of changing the habits of technology use among 
members of an organization. 

One of the main ways a change-agent can engender such change is through an effective 
PD program. But what does effective mean? And in the context of this study, what does 
effective technology-related PD mean? These questions are necessary because it is impos¬ 
sible to measure the efficacy of a program for change on its target audience, or the impact 
of the change-agent, without first examining the elements of the program itself. 

Therefore, this study examines the professional development program that was imple¬ 
mented by the change-agent at the university. Specifically, it will determine whether or 
not the PD initiatives that were implemented adhered to best practice criteria posited in 
the literature, and then analyze the impact the PD initiatives had on the use of Moodle 
among faculty. 

The theoretical model used is Everett Rogers' Diffusion of Innovations model. This model 
is a seminal work in diffusion theory with regards to technology use in organizations. 
According to Rogers' model, a change-agent is among the five most important facets of 
106 increasing the use of a form of technology within an organization. 
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Literature review 

Rogers’ (2003) Diffusion of Innovations (DOl) model states that there are five main vari¬ 
ables which most influence the use of an innovation in an organization, one of which is 
the Extent of the Change-agent's Promotion Efforts. This means that a change-agent who is 
employed to facilitate technology-related change is reported to be one of the most impor¬ 
tant elements in successfully engendering change among its users. According to Rogers 
(2003), a change-agent seeks to shift clients - in this case teachers - away from a position 
of reliance on the change-agent, to one of self-reliance. One of the most common ways a 
change-agent can do this in the context of education, including English Language Teaching 
(ELT), is by way of teacher education, or a specifically designed Professional Development 
(PD) program. 

In order to do that, research (Ensminger & Surry, 2008; Hall & Hord, 1987; Rogers, 2003) 
indicates that when a technology-related PD program is implemented, the change-agent 
should adhere to a number of essential criteria. For example, the duration of PD initiatives 
is repeatedly listed as highly important in successful PD (Corcoran 1995; Crawford, 2003; 

Garet & Porter, 2001; Hiebert, 1999; Villegas-Reimers, 2003; Wells, 2007). Duration refers 
to how ongoing a PD program is, rather than how long specific events such as workshops 
run for. In addition, the need for relevant, active PD initiatives such as workshops and ori¬ 
entation sessions that engage the participants is commonly cited (Coggins, 2008; Corcoran, 

1995; Crawford, 2003; Dias & Atkinson, 2001; Garet & Porter, 2001; Hiebert, 1999; Wells, 

2007; Zemelman, Daniels, & Hyde, 2005). A third criterion of successful PD, according to 
the literature, is support offered to participants by change-agents, including follow-up sup¬ 
port after the implementation of the PD initiatives (Coggins, 2008; Crawford, 2003; Wells, 

2007; Villegas-Reimers, 2003). Research also included the need for a support group that 
is available to help educators in relation to technology-specific issues. Moreover, on-site, 
locally situated PD events that are organized and implemented by a change-agent is also 
commonly listed (Coggins, 2008; Chapelle, 2001; Corcoran, 1995; Crawford, 2003; Villegas- 
Reimers, 2003). 

In addition, Havelock's model of change (1995) states the necessity of creating and using 
appropriate materials and resources within the context of a PD program. Havelock says 
that such materials should help to facilitate the change process without disrupting clients 
from their normal routines. He states that a variety of resources necessary to address the 
organization's needs can be used, and that they can come in many forms, including person¬ 
nel, informational, and material. 

Further criteria necessary for implementing successful change in use of technology 
relate to communication. Rogers (2003) states that a change-agent’s success in enhanc¬ 
ing the use of an innovation, is positively related to the extent of a change-agent’s efforts 
in contacting an organization's members. Therefore, it is vital that clients (in the context 
of this study the clients were the university's faculty members) are informed about any 
changes that are made, and about how they may be introduced to any changes and helped 
in any lurid of institutional transformation. Havelock (1995) also says that contacting an 
organization's members is an important stage for the change-agent to address during the 
change process. 

This brief overview of the literature indicates that a number of criteria must be adhered 
to if change-agents want to successfully facilitate technology-related change within an 
organization. The literature suggests that if a change-agent adheres to the recommended 107 
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criteria during the implementation of a technology-related PD program, then the chances 
of successfully changing the technology use of clients will be enhanced. This study will 
examine whether or not the change-agent adhered to the criteria. 

Method 

Participants 

For this study, the subjects were faculty members working at the university when the PD 
program was implemented and made available. The total number was approximately 45. 
Although all 45 participants came from the university in Japan, different numbers of par¬ 
ticipants were used during the different data collection phases. 

Data collection 

This study utilized a mixed-methods approach to collect the data. The different methods 
used during the data collection phase of this study are described below. 

Online survey 

An online survey was used to collect data relating to the impact of the PD initiatives. The 
survey was sent to 45 faculty members, and 42 responded. The survey was created using 
www.esurveyspro.com. It was designed so that when participants opened their email, they 
had to click on an embedded link, which then redirected them to the survey. 

Semi-structured individual interviews 

The follow-up individual interviews were used to provide more detailed insights into 
responses that came from the online survey, and to get a richer understanding of the 
impact of the Moodle-related PD initiatives. The interviews were semi-structured. According 
to Nunan (1992), in a semi-structured interview, the interviewer has a general idea of where 
he or she wants the interview to go, but does not enter the interview with a complete set 
of predetermined guestions. 

The participants for the face-to-face interviews were selected from the respondents to 
the survey. An advantage of the semi-structured approach lies in the interviewer being able 
to clarify guestions if participants are unsure of the meaning, and probe for elaboration 
(Creswell, 2005). Participants were given the questions before the interviews so they could 
become familiar with the proposed topics (Merriam, 1998). 

The face-to-face interviews were conducted as exploratory conversations rather than ses¬ 
sions where asking questions may have seemed as though specific, or contrived, answers were 
being searched for. Merriam (1998) argues that where a lack of theory exists to explain a current 
phenomenon, such as factors which influence faculty use of Moodle in an EFL context in Japan, 
a flexible, qualitative approach best enables the researcher to conduct research inductively. 
The participants for the face-to-face interviews were selected from the respondents to the 
online survey. As much as possible, the selected participants were chosen to represent a 
wide range of Moodle-user, as well as a broad cross-section of experience in the field of 
108 ELT, including experience in Japan. It was hoped that by including respondents with a wide 
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variety of backgrounds, experience in ELT, and Moodle-user habits, their answers would be 
representative of the wider Moodle-user population and their answers relevant to a wider 
audience.A full breakdown of the selected participants can be seen in Table 1. 


Table 1: Participant information for face-to-face interviews 


Identifier 

Age 

Finished 

Masters 

course 

Years 

teaching 

Years 

teaching at 
uni level 

Year at 

university 

Moodle 

use 

Member 

technology 

committee 

1-1 

31-35 

2006-8 

7-10 

4-6 

3rd 

Never 

No 

1-2 

35+ 

2001-5 

7-10 

4-6 

3rd 

Never 

No 

1-3 

31-35 

2001-5 

7-10 

1st year 

1st 

In most 

classes 

No 

1-4 

26-30 

2001-5 

4-6 

1-3 

3rd 

Never 

No 

1-5 

35+ 

2001-5 

4-6 

1-3 

3rd 

Never 

No 

1-6 

31-35 

2006-8 

4-6 

1st year 

1st 

Never 

No 

1-7 

26-30 

2006-8 

7-10 

1st year 

1st 

In some 

classes 

No 

1-8 

31-35 

1996-2000 

7-10 

7-10 

4th 

In all 

classes 

Yes 


Analysis 

The following sections provide information about the methods of analysis used with the 
different forms of data that were collected, in relation to each of the guestions in the study. 

The first guestion of this study examined whether or not the PD program implemented 
by the change-agent adhered to technology-related PD best practice procedures. In order 
to do so, an original, best-practice rubric was created that contained a number of best 
practice criteria, drawn from a wide variety of scholars who had implemented effective 
technology related change at their respective institutions, especially in relation to teacher 
use of technology. These scholars have had their exemplary change programs featured in 
works such as Developing Faculty to Use Technology, Teaching Faculty Flow to Use Technology; 
and Teacher Education in CALL. The specific nature of the change effected by these specific 
scholars is available for further examination by looking at the articles provided in the 
references section at the end of this paper. In selecting best practice criteria for inclusion 
in the rubric, only those technology-related PD criteria that were mentioned by scholars 
and experts more than four times throughout the literature review were included. The 
decision to use four as the cut-off point was arbitrary, and was done primarily to ensure 
that only the most often discussed technology-related PD criteria were utilized. In limit¬ 
ing inclusion in the rubric to certain PD criteria that were mentioned four or more times 
by different scholars, nine criteria were used. They included Support; Theory Workshops; 
Relevance; Learner-centered; Manuals and how-to guides; Authentic examples of use; Access; 
Incentives; and Duration. 

In addition, certain criteria were also used in deciding which scholars and experts 
to include in the best practice rubric. Best practice definitions state the need to utilize 
the expertise of people in a field that may have significant experience and proven suc¬ 
cess. Therefore, in creating the rubric used to analyze the PD initiatives in relation to best 109 
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practice criteria, the educators and scholars selected for inclusion were all leaders in the 
Held of educational technology, or technology-related PD, who had implemented successful 
PD programs at their educational institutions. The rubric can be seen in Table 2. 


Table 2: Technology-related PD best practice rubric 








Manuals/ 




Name, 


Theory/ 

Work¬ 

Relevant/ 

Learner 

how-to 

Authentic 

(Online) 


year 

Support 

pedagogy 

shops 

practical 

centred 

guides 

examples 

Access 

Incentives Duration 

Coggins, 

2008 

X 





X 

X 

X 


Clark, 

2003 


X 

X 


X 

X 


X 


Clark, 

2006 

X 




X 



X 

X 

DeVry, 

2003 

X 



X 



X 


X 

Egbert, 

2006 


X 



X 




X 

Gamble, 

2003 

X 



X 


X 


X 


Grant, 

1996 

X 



X 

X 



X 

X 

Wells, 

2007 

X 

X 


X 

X 




X 

Mathis, 

2003 

X 




X 

X 

X 

X 


Pollock et 
al, 2001 

X 




X 

X 

X 

X 


Robinson 

Pearce, 

2003 

X 


X 

X 

X 


X 


X 

Moore, 

2001 

X 


X 

X 




X 

X 

Epper, 

2001 

X 



X 

X 




X X 

Hartman, 

2001 

X 


X 

X 

X 



X 

X 

Hutchison, 

2001 

X 


X 


X 



X 

X X 

Rickard, 

2006 


X 

X 

X 

X 



X 

X 

Wong & 
Benson, 
2006 


X 


X 


X 



X 

Robb, 

2006 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Schwartz 
& Phillips, 
2003 

X 

X 

X 


X 


X 

X 

X 


Online survey 

Analysis of the online survey was carried out using SPSS. In the second guestion of this 
study. What impact did the PD initiatives have on faculty use of Moodle?, descriptive statis¬ 
tics were used to determine freguencies, means, and standard deviation for the construct 
110 Impact of the PD initiatives, containing seven items. In addition, a Spearman’s Rank Order 







Stanley: Impact of a change-agent on faculty use of technology 


correlation test was used to test for relationships between the dependent variable How often 
did you use Moodle?, and the construct Impact of the PD initiatives, containing seven items. 

Semi-structured individual interviews 

In analyzing the individual interviews, a number of technigues were used to look for key¬ 
words that expressed major concepts or ideas and allowed the researcher to classify the 
responses into emerging themes. Following the collection of data from the online survey, 
eight faculty members were interviewed. Upon completion of the interviews, the recorded 
audio was transcribed verbatim using Microsoft Word, each in different documents, accord¬ 
ing to the faculty member that was interviewed. 

Results 

Question 7 : Did the PD initiatives that were implemented by the change- 
agent adhere to best practice criteria? 

An analysis will be provided here to determine if the PD program implemented in this 
study adhered to technology-related best practice recommendations. The PD initiatives that 
were utilized will be compared with, and measured against, the criteria in the best practice 
rubric illustrated in Table 2. 


Support 

Fifteen of the nineteen PD experts cited in the rubric indicated that having a "support net¬ 
work" on hand to help faculty with issues related to technology was of utmost importance. 
Wells (2007) noted that long-term, continuous pedagogical, social, technical assistance 
was important in the implementation of successful, technology-related PD. The support 
network that existed at the university in this study consisted of six members, all of whom 
were Moodle users and comfortable in explaining and supporting different features of 
Moodle. Further, different support members had offices in different buildings, meaning 
that faculty had easy access to any technology support member, regardless of where their 
office was located. If faculty had to troubleshoot something, or felt they needed immediate 
on-campus support, it was readily and easily accessible. 

Workshops 

Eight of the CALL scholars whose work was drawn upon in the rubric stated the importance 
of using workshops to help teachers effectively learn about technology (Clark, 2003; Egbert, 
2006; Robb, 2006; Wells, 2007). However, in examining the workshops implemented by 
the change-agent in this study analysis indicated that they also met other important best 
practice criteria. The workshops were held once a week for an entire semester. The most 
convenient time was decided upon by an anonymous faculty vote. They were conducted by 
members of the university's technology research committee and in each workshop there 
was a support network of 5-6 expert Moodle users available to help any faculty partici¬ 
pants with problems or guestions they may have had. The workshops were voluntary and 
participation was not mandated. 


Ill 




The JALT CALL Journal 2015: Regular Papers 


The workshops adhered to the "support network" criteria, the most consistently referred 
to criteria in the rubric. The change-agent was responsible for running the workshops, 
and also for ensuring that relevant expert Moodle users were available for the workshops. 
Some committee members were more adept with some of Moodle’s features than others, 
so the agent had to ensure that all support network members who were proficient in a 
particular Moodle feature were available for the workshop that matched that feature. If 
faculty members could not attend, or wanted to view the contents of the workshops later, 
everything was made available online, and faculty was notified of this after each workshop. 

Relevant/practical and learner-centered 

Eleven of the scholars used in the rubric referred to the importance of relevant/practical 
activities, and fourteen referred to the importance of learner centered PD activities. The 
PD workshops that were implemented by the change-agent illustrate how relevant and 
practical they were to faculty, as well as how they took a learner-centered approach. The 
workshops typically followed a 3-step process. The first involved a brief explanation of 
the technical aspects of the relevant Moodle feature being presented on the day, which 
mirrored what was in the freely available video-based and MS Word-based instructional 
manuals. Following that, two or three expert Moodle-users from the attendant support 
network would demonstrate how they were using that particular Moodle feature in their 
classes and explain the rationale behind it. And finally, participating faculty were given the 
option to practice using the feature in any way they wanted and to ask guestions in relation 
to the specific Moodle feature. The workshops went for an hour, and participating faculty 
were sent a follow-up email to check on any issues they may have had. The workshops 
enabled participating faculty to see how expert users were using Moodle in their classes 
and a chance to experiment with ideas that they might realistically use in their own classes. 

How-to manuals 

From the rubric, seven people recommended that "how-to" documents in some form be used 
in Professional Development initiatives for faculty (Clark, 2003; Mathis, 2003; Robb, 2006; 
Wong & Benson, 2006). The how-to video guides were created by the change-agent and 
the technology committee overseen by the change-agent, and uploaded to the university’s 
Moodle site. They were made available in two formats - a lower guality version available 
for immediate viewing online and a higher guality version that could be downloaded and 
saved to any user's computer. As long as faculty had an internet connection, then they 
could access both versions anywhere, anytime, regardless of their location, as the univer¬ 
sity’s Moodle site was online. Moreover, The MS Word-based instructional manuals were 
designed to accompany and add to the video-based manuals described above. The MS Word 
manuals covered the same issues and points addressed in the videos. Faculty could access 
them online, or download them and keep hard copies. 

Authentic examples of Moodle use in teaching practice 

For each Moodle feature, or module as they are called in Moodle, such as Forums, Journals, 
Blogs, and Wilds, there would be authentic, step-by-step examples of how that particular 
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Moodle feature was being used by expert Moodle users in classes at the university. That 
allowed faculty members to see how to apply Moodle in real classes. 

Access 

Thirteen of the scholars used in the best-practice rubric place importance on access to PD 
materials. All of the help materials were on the university's Moodle site. So as long as 
faculty had an internet connection, then they could access all help materials anywhere, 
anytime, regardless of their location. On campus, faculty also had 9-5 access to the Moodle 
support committee. 

Duration 

Seven scholars state the importance of duration in successful technology-related PD. 
Further, they all say that the longer the PD is available, the better. This does not mean that 
the duration of specific events, such as workshops, must be longer, rather they should be 
ongoing on a weekly basis for interested faculty members, and available on an ongoing 
basis on a variety of platforms and mediums. In short, one-off PD sessions are not recom¬ 
mended. This is primarily to ensure that faculty members can access the PD initiatives at 
any time, at any place, even if they have no time to attend specific events such as workshops. 
The workshops for faculty in this PD programme ran for an entire semester, totalling 15 
weeks. In addition, the help Hies that were uploaded were permanently available to faculty 
on the Moodle website. Finally, the support network on campus was available at all times 
during working hours, so there was no time that faculty members at the university could 
not make use of some sort of technology related PD facility. 

Summary 

By comparing the PD program with the best-practice rubric, it is clear that almost all of the 
most commonly cited criteria were adhered to during the implementation of the program. 
According to the literature, there was an implication that should these most cited criteria 
be adhered to in a technology-related PD program, then the effect on members of an edu¬ 
cational organization would lead to increased use of technology, or specifically targeted 
technology. The next section will examine how effective the PD program was and whether 
it had a positive influence on faculty. 

Question 2: What impact did the PD initiatives have on the use of Moodle 
among Faculty? 

Based on the literature, there was a belief that good PD would not only lead to increased 
awareness of Moodle as an innovation, but a concomitant increase in use among faculty 
members. However, survey results for the guestion How often do you use Moodle 1 indicate 
that that was not the case at all. Figure 1 indicates guite clearly that, despite the implemen¬ 
tation of the PD initiatives, actual use of Moodle among faculty members did not increase 
or change very much. Indeed, more than half of the 42 faculty members who answered this 
guestion never used Moodle at all. 


113 
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Figure 1. Use of Moodle among faculty members 


This section explores the specific influence that the various PD initiatives had on faculty, 
and discusses issues relating to the unanticipated results. 

The following section provides results for Question 2 What impact did the PD initiatives 
have on the use of Moodle among faculty? Data was collected using an online survey and 
interviews with selected participants. 

Cron bach's Alpha test of reliability 

A Cronbach’s alpha test of reliability was performed on the construct Impact of the PD initia¬ 
tives in order to examine the internal reliability of the construct and the items contained 
within. Brown (1998) says that "one fundamental concern in measuring anything is that 
the results should be the same every time you measure it” (p.121). Brown also says that 
one of the most common ways of testing for internal reliability in surveys is by way of 
Cronbach's alpha. The closer Cronbach’s alpha coefficient is to 1.0, the greater the internal 
consistency of the items (Gliem & Gliem, 2003). George and Mallery (2003) provide the 
following rules of thumb: "a > .9 - Excellent, a > .8 - Good, a > .7 - Acceptable, a > .6 - 
Questionable, a > .5 - Poor, and a < .5 - Unacceptable” (p. 231). Nunnally (1978), also says 
that a Cronbach's alpha score of 0.7 - 0.8 or above is an acceptable indication of internal 
reliability, as does Field (2009). 

Cronbach's alpha scores for the construct Impact of the PD initiatives (a = .922) were well 
above the 0.7-0.8 threshold recommended in the literature (Field, 2009; George & Mallery, 
2003; Gliem & Gliem, 2003; Nunnally, 1978; Pallant, 2001) Therefore, the construct Impact of 
the PD initiatives was considered to be internally reliable, and all items considered relevant. 
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Descriptive statistics 

Table 3: Descriptive statistics for the construct Impact oF the PD initiatives 



N 

Mean 

SD 

The Moodle-related PD initiatives taught me more about Moodle. 

33 

4.03 

.984 

The Moodle-related PD initiatives gave me new ideas for my classes. 

33 

3.67 

1.164 

The Moodle-related PD initiatives gave me new perspectives on using 
Moodle in my teaching. 

33 

3.79 

1.023 

The Moodle-related PD initiatives made me feel less anxious about 
using Moodle in my classes. 

33 

3.52 

.906 

The Moodle-related PD initiatives enhanced my development as a 
teacher. 

42 

3.45 

.942 

The Moodle-related PD initiatives led me to take-up, and/or increase 
my use of Moodle. 

42 

2.71 

1.419 

The Moodle-related PD initiatives influenced my use of Moodle. 

42 

2.90 

1.376 


Each item in the online survey was positively framed. In four items, 33 faculty members 
responded. Nine participants were omitted from further analysis on these particular items 
because they selected the "Never Used" option in the survey when asked if they had used 
specific elements of the Moodle-related PD initiatives. The results indicated that the impact 
on the 33 teachers who used specific PD initiatives was generally positive - the mean score 
of those who made use of the PD initiatives ranged from x = 3.52 to x = 4.03 (3 = Neutral, 

4 = Agree) for the five relevant items. This indicated that for those faculty members who 
made use of specific Moodle-related PD initiatives, such as how-to guides and workshops, 
the impact on them leant more towards “Agree" than “Neutral." It was interesting to note 
that for the two items that related specifically to use and increased use of Moodle - and 
which all participants had to answer - the mean score was much lower. For example, the 
item The Moodle-related PD initiatives led me to take-up, and/or increase my use of Moodle 
had a mean score of 2.71 (2 = Disagree, 3 = Neutral), while the item The Moodle-related PD 
initiatives influenced my use of Moodle' had a mean score of 2.90 (2 = Disagree, 3 = Neutral). 

This showed that, regardless of whether faculty members made use of the PD initiatives or 
not, the initiatives did not really lead to increased use of Moodle among faculty. 

Spearman's Rank Order Correlation (SPSS) 

A Spearman’s Rank Order correlation was run in SPSS to identify any relationship between 
a) How often do you use Moodle in your classes? and b) The impact of the PD initiatives (con¬ 
taining 7 items). The results can be seen in Table 4, overleaf. 

There was a significant, positive correlation between faculty use of Moodle and the impact 
of the PD initiatives on 6 of the 7 items. Four items had significant correlation at the 0.01 
level (2-tailed), and 2 items had a significant correlation at the 0.05 level (2-tailed). Only 1 
item had no significant correlation - The Moodle-related PD initiatives made me feel less anxious 
about using Moodle in my classes (r s = .303, P = .086). It was interesting to note that two of 
the four items which showed significant correlation at the 0.01 level related to how the PD 
initiatives taught faculty more about Moodle and gave them new perspectives on using it. 115 
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Table 4: Spearman's Rank Order correlation statistics for the construct 'Impact of the PD 
initiatives' 


How often did you use Moodle in your classes? 

Spearman's 

Rho Correlation 

Coefficient 

Sig. (2-tailed) 

N 

The Moodle-related PD initiatives taught me more about 
Moodle. 

.484** 

.004 

33 

The Moodle-related PD initiatives gave me new ideas for 
my classes. 

.379* 

.030 

33 

The Moodle-related PD initiatives gave me new 
perspectives on using Moodle in my teaching. 

.523** 

.002 

33 

The Moodle-related PD initiatives made me feel less 
anxious about using Moodle in my classes. 

.303 

.086 

33 

The Moodle-related PD initiatives enhanced my 
development as a teacher. 

.344* 

.026 

42 

The Moodle-related PD initiatives led me to use, and/or 
increase my use of Moodle. 

.735** 

.000 

42 

The Moodle-related PD initiatives influenced my use of 
Moodle. 

.513** 

.001 

42 


**. Correlation is significant at the 0.01 level (2-tailed). 
*. Correlation is significant at the 0.05 level (2-tailed). 


Results From semi-structured individual interviews 

Two faculty members who were interviewed (I- 3 , 1 - 7 ) said the PD initiatives had a positive 
influence on them, including their adoption and use of Moodle. One teacher (I- 7 ) said that 
he used the how-to manuals a lot to help him set up courses and get new ideas from other 
teachers who were using Moodle. He said, “The manuals were well made and useful. They 
were good because there was a document and video manual. 1 used the document more 
because I could download it, but 1 always watched the video first." The other teacher (I- 3 ) 
who was positively influenced by the PD initiatives said that the PD initiatives, particularly 
the workshops, helped him with ideas in his teaching practice. He said the PD initiatives 
helped him use Moodle more efficiently and creatively. 

1 would have tried different things myself, but the PD initiatives helped me with ideas. 1 
reached a point during my first semester where I thought f d hit a wall with Moodle and 
wasn't sure how practical it was going to be. Then 1 went to a workshop and it impacted 
on me and my use because I realized Moodle had a broader use and there were lots of 
things 1 could do with it. You can get stuck on simple things and get frustrated and then 
leave it alone, but seeing other ideas helps you to think of other ways yourself. 

The same teacher (I- 3 ) said that he also made constant use of the technology support com¬ 
mittee. He believed that the ease in which he could consult committee members on the 
run made using Moodle easier, as he did not have to sift through how-to documents or 
send email inquiries. He said, "1 often tried to teach myself, but if I ran into a brick wall, it 
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was nice to be able to pop next door or see a committee member and quickly ask them a 
question. Having that availability was convenient for my needs.” 

Another teacher (I-5) said that the PD initiatives had reduced any anxiety that may have 
been involved in using new technology. The PD initiatives helped with his use of Moodle in 
subsequent months because he was already much more familiar with Moodle than some 
other teachers who had not made use of the PD. He also said that as he had started to use 
Moodle, the PD initiatives had been helpful. He said that he had not had to start from “0" 
but from a 2-3 (5 being completely familiar with Moodle). 

Two teachers (I-i, I-4) said that the PD initiatives had no impact on their uptake or use 
of Moodle whatsoever. One teacher (I-i) said that he taught himself most basic things, so 
the video manuals he saw did not teach him anything he did not already know. He already 
knew how to do most things. One main reason he gave for being familiar with Moodle was 
that other people in his committee had taught him about Moodle, and that he had learnt 
by copying what other teachers had been doing with Moodle in the same class. The other 
teacher (I-4) said that he never used Moodle, and switched off whenever conversations 
turned to Moodle. Therefore, the PD initiatives had no impact on him because he never 
used them or Moodle. 

Discussion 

How did the PD initiatives influence Faculty in their use of Moodle? 

There has been an implication presented in the literature that if best practice procedures 
are followed, the ensuing impact on technology use by teachers may well be positive and 
enhanced. A perfect example of this comes from Ertmer (1999), who said that faculty use 
and uptake of technology is influenced by a number of factors, which she referred to as 
"enablers" and “barriers." She said "enablers" include such things as access to hardware, 
guality software, the Internet, technical support, as well as administrative and peer sup¬ 
port, and that it is likely that either a decrease in barriers, or an increase in enablers, would 
lead to greater technology use. Egbert (2006) added further weight to this idea when she 
said that teacher change and growth occur through learning that is situated in classrooms. 
The PD rubric that was created to analyze the PD initiatives implemented in this study also 
drew upon technology-related PD scholars, and used a number of criteria which suggested 
would have a positive impact on PD participants. Generally, the implication was that if PD 
initiatives followed the ideas of Egbert (2006), Ertmer (1999), and those scholars used in 
the rubric, then the impact on teacher uptake, and use of technology (Moodle), would be 
positive and noticeable. 

In this study, it was demonstrated that technology-related best practice procedures were 
used in accordance with recommendations made in current PD literature. However, the 
results here seemed to indicate that the PD initiatives did not necessarily have the impact 
that the literature suggested they might. In looking at the results from the various sources 
of data, it was not immediately clear what kind of influence the PD initiatives had on faculty. 
In terms of simple numbers, the number of faculty members who utilized the PD initia¬ 
tives and introduced themselves to Moodle definitely increased. So in that sense, the PD 
initiatives would appear to have been successful, and the ideas presented in the literature 
regarding educational technology best practice procedures correct. On the other hand, the 
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nature of how the PD initiatives were used, and the amount that Moodle was used in terms 
of modules and activities, tended to contradict the literature. 

In examining the item from the online survey the Moodle-related PD initiatives taught me 
more about Moodle, the mean score of 33 respondents was x = 4.03 (4 = Agree). Further, in 
the Spearman’s Rank Order correlation test, the correlation between the items How often 
didyou use Moodle in jour classes? and the Moodle-related PD initiatives taught me more about 
Moodle was significant at the 0.01 level (0.004). This suggested that faculty did learn more 
about Moodle through the PD initiatives, and that they led Moodle users to increase their 
understanding of Moodle. However, it must be noted that the item the Moodle-related PD ini¬ 
tiatives taught me more about Moodle had a "Never Used" option, meaning that 9 participants 
of the 42 who took part in the online survey did not answer this item. This meant that the 
results related only to users who were already using Moodle and who made use of Moodle 
and the PD initiatives. So, in essence, the initiatives were effective on an audience that 
was already captured, but not on the more targeted audience of non-users of Moodle. One 
comment by a consistent Moodle user (I-3) during the face-to-face interviews illustrated 
this point. The respondent said that the PD initiatives were particularly helpful for him 
after he found that he could not think of any more original or unigue ways to use Moodle. 

However, a comment made by a non-Moodle user also demonstrated that the PD initia¬ 
tives were not as successful with faculty who did not use Moodle. The respondent (I-4) said 
that he never used Moodle and switched off whenever conversations turned to Moodle. 
Therefore, the PD initiatives had no impact on him because he never used them, or Moodle. 

Thus, one could argue that, as Egbert (2006) suggested would happen, some faculty 
growth and understanding may have occurred as a result of the PD initiatives, which was 
backed up by the mean score of x 4.03 (4 = Agree) for the item the Moodle-related PD initia¬ 
tives taught me more about Moodle. However, these developments did not lead to increased 
use of Moodle or a significant change in teacher habits for non-users of Moodle. 

This was clearly illustrated by results from the item The Moodle-related PD initiatives 
led me to take-up, and/or increase my use of Moodle, which had a mean score of x .71 (2 = 
Disagree, 3 = Neutral) for 42 users. This indicated that the actual influence of the PD initia¬ 
tives was not as significant as may have been expected. Moreover, in the Spearman’s Rank 
Order correlation test, the correlation between the dependent variable. How often didyou 
use Moodle in your classes? and the independent variable the Moodle-related PD initiatives 
taught me more about Moodle was significant at the 0.01 level (0.000). The significance of 
this positive correlation (correlation coefficient .735**) suggested that faculty who did 
use Moodle, increased their use of Moodle, while faculty who did not use Moodle, did 
not increase their use of Moodle or take it up. Therefore, it seemed once again that the 
PD initiatives were particularly successful with faculty already using Moodle, but not so 
influential on faculty who were not using Moodle - ostensibly the audience who the PD 
initiatives were designed to target. 

The results above referred to the PD initiatives as a whole, but did not focus on any spe¬ 
cific initiative. However, if one specific PD initiative was focused on, the results were much 
the same, which is a result that was somewhat different from what the literature suggests. 
For example, in focusing on workshops, Brzycki and Dudt (2005) made the implication that 
if workshops paid sufficient attention to participants' specific needs and desires concerning 
technology integration, provided practice in transferring skills to participants’ own situ¬ 
ations, and provided ongoing technical assistance, then the results would be positive and 
118 that the workshops would meet their objectives. However, results from the workshops that 
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were implemented as part of the PD initiatives indicated that the impact was not as success¬ 
ful as anticipated. While the perceptions of faculty members toward the workshops were 
positive, the item you will introduce ideas from this workshop into your classes consistently 
showed the lowest mean score of around x 3-3.5 (3 = Neutral, 4 = Agree) across the eight 
different Moodle-related workshops. Therefore, the influence of the workshops on faculty 
use of Moodle was not as successful as best practice literature suggested it may have been. 

One thing that came from the face-to-face interviews that had not been accounted for, 
or found in results from the online survey, was the indirect impact that the PD initiatives 
had on faculty. Through talking to a number of interview participants, it became obvious 
that even though a number of them had made very little first-hand use of the PD initiatives, 
their use of Moodle had been indirectly influenced by them. For example, one teacher ( 1 - 6 ) 
stated at the time of the study that he had never made use of the PD initiatives or been 
influenced by them, ffowever, when he began using Moodle after the implementation of 
the PD program, he set up his course on Moodle on the advice and instruction of another 
faculty member (I-5) - one who had made use of the PD program. So the teacher ( 1 - 6 ) who 
had said that the PD initiatives had not influenced him at all had in fact been influenced 
by them, albeit in a more indirect fashion. This was also the case with I-i. ffe said that he 
had learnt most things from Moodle by himself, but had also learnt from other faculty 
members, who, incidentally, were part of the technology committee designed to help fac¬ 
ulty with their use of Moodle. So even though he stated that the PD initiatives had had no 
impact on him, he was in fact influenced by them via the face-to-face support he received by 
technology committee members. This may well have continued to happen over the ensuing 
years, or at least while faculty members were employed by the university. However, such 
a longitudinal study was impossible in this context because almost all teachers were on 
limited term contracts. This is an issue that needs further explanation in the context of 
technology related ELT in Japan 

In the context of this study it seems that, despite the extent of the change-agents' pro¬ 
motion efforts in implementing the various PD initiatives, they did not have as much of 
an influence on the use of Moodle among faculty as might have been anticipated. It has 
been shown that a number of recommended best practice criteria in relation to technology- 
related PD were utilized during the implementation of the PD program. Therefore, it was 
not unreasonable to anticipate that the impact of the PD program would be significant. 
However, results indicated that the desired impact of Moodle use among faculty was rather 
underwhelming. Therefore, perhaps there were other variables that were far more influen¬ 
tial on the adoption and use of Moodle. 

Conclusion 

This paper examined a technology-related PD program that was implemented at a language 
university in Japan. It looked at the elements of the program itself, and then at the impact 
that the program had on its target - faculty members within the university. Prior research 
had indicated that if PD initiatives were implemented in adherence to certain best prac¬ 
tice guidelines, then they would most likely have a positive impact on use of technology 
among members of an organization. However, results here indicated that the influence of 
the PD initiatives was mixed. They had a positive impact on faculty members who were 
already using Moodle, but they did not have as much influence on faculty members who 
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were not users of Moodle - ostensibly the people that the PD initiatives were designed to 
influence most 

A positive aspect of the PD initiatives that had not been accounted for was the indirect 
impact that they had on some faculty. A number of faculty members noted that they had 
learned about Moodle not by the PD initiatives themselves, but by other faculty members 
who had learned about Moodle through the PD initiatives. Thus, there was a kind of second¬ 
hand learning, or learning that was passed down the line. This was positive because it indi¬ 
cated that information relating to an innovation can be diffused throughout an organiza¬ 
tion in different ways, as long as a good PD program is put in place. However, a limitation 
of doing research in the context of an institution with constantly changing personnel is 
that ongoing efficacy is almost impossible to measure. 

Nonetheless, the key issue was no doubt the fact that the change-agent, in the context 
of Rogers' diffusion model (2003) and best practice literature, had done what was reguired 
and recommended according to best practice literature, yet the anticipated result was not 
forthcoming. This suggested that perhaps the efficacy of a PD program implemented by a 
change-agent is not always as immediately obvious as may have been predicted. Moreover, 
it suggested that perhaps the five variables Rogers (2003) lists as being most influential 
on the use of an innovation may not have egual influence and that some variables are far 
more influential than others. Further research may well indicate that of the five variables 
Rogers includes, some may even counteract the influence of others. 
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